java -jar rhq-env-provisioner.jar rhq-perf-env.properties
This tool would do the following:
(optional - probably a v2 feature) download a PostgreSQL distribution from the specified URL and use SSH to push it to the specified machine and install it there
download an RHQ distribution from a specified URL (could be a local file, a final release from SourceForge, or a nightly build from Hudson)
via SSH, push the distribution to the specified machines and install RHQ Servers on those machines using our automated/silent Server install mechanism; the rhq-server.properties used would be provided to the provisioning tool and template vars would be realized for
Server-specific props such as the Server IP
start the Servers in headless mode via SSH
download the Agent distribution from one of the Server's download page
install n Agent instances to the specified machines by either a) invoking our agentcopy tool via SSH or b) calling our RemoteInstallManagerRemote SLSB
start the Agents in headless mode via SSH
wait for Agent autodiscovery to complete then use CLI script to import all discovered Resources
(optional) create a bunch of compatible and mixed groups containing the inventoried Resources
(optional) create a bunch of test roles and users, with varying levels of authorization
execute SQL and/or CLI script to populate the DB with fake past data for all
metrics - 7 days worth of raw data for all enabled metrics - use Greg's DataInserter class as a starting point
avail - n days worth of ups and downs
calltimes - n days worth of calls
events - n days worth of events
(optional/v2) - plugin config updates, resource config updates, operation executions
The tool would initially have a command-line interface but a Web interface could be added later.
The tool could be used for provisioning environments for a variety of purposes:
large-scale perf testing
demos
day-to-day dev testing
To configure an HA perf environment with 2 Servers and 2 Agent machines running 50 Agents each, the tool would be invoked as follows:
java -jar rhq-env-provisioner.jar rhq-perf-env.properties
And rhq-perf-env.properties would look something like:
### RHQ Server Provisioning Specs
serverDistributionZipUrl = https://hudson.example.org/hudson/job/rhq-master/lastSuccessfulBuild/artifact/modules/enterprise/server/container/target/rhq-server-4.1.0-SNAPSHOT.zip
server[0].host = perfserver1.example.org
server[0].sshUser = rhq
server[0].sshKeyPassword = rhq
server[0].serverInstallParentDir = /home/rhq/perf-env
server[0].env.RHQ_SERVER_JAVA_HOME = /usr/java/jdk1.6.0_25
server[0].env.RHQ_SERVER_MAX_HEAP_SIZE = 2000M
server[0].env.RHQ_SERVER_MAX_PERM_SIZE = 400M
server[1].host = perfserver2.example.org
server[1].sshUser = rhq
server[1].sshKeyPassword = rhq
server[1].serverInstallParentDir = /home/rhq/perf-env
server[1].env.RHQ_SERVER_JAVA_HOME = /usr/java/jdk1.6.0_25
server[1].env.RHQ_SERVER_MAX_HEAP_SIZE = 2000M
server[1].env.RHQ_SERVER_MAX_PERM_SIZE = 400M
### RHQ Agent Provisioning Specs
agentMachine[0].host = perfagent1.example.org
agentMachine[0].sshUser = rhq
agentMachine[0].sshKeyPassword = rhq
agentMachine[0].agentInstances = 50
agentMachine[0].agentPortRangeStart = 27000
agentMachine[0].agentPortRangeIncrement = 50
agentMachine[0].agentInstallParentDir = /home/rhq/perf-env
agentMachine[0].agentNamePrefix = perfagent1-
agentMachine[0].agentEnv.RHQ_AGENT_JAVA_HOME = /usr/java/jdk1.6.0_25
agentMachine[0].agentEnv.RHQ_AGENT_JAVA_OPTS = -Xms150M -Xmx150M -Djava.net.preferIPv4Stack=true
agentMachine[0].agentEnv.RHQ_AGENT_ADDITIONAL_JAVA_OPTS = -Don.perftest.scenario=configurable-5 -Don.perftest.server-d-count=3 -Don.perftest.service-d-metrics-count=120 -Drhq.trace-command-config=true
agentMachine[1].host = perfagent2.example.org
agentMachine[1].sshUser = rhq
agentMachine[1].sshKeyPassword = rhq
agentMachine[1].agentInstances = 50
agentMachine[1].agentPortRangeStart = 27000
agentMachine[1].agentPortRangeIncrement = 50
agentMachine[1].agentInstallParentDir = /home/rhq/perf-env
agentMachine[1].agentNamePrefix = perfagent2-
agentMachine[1].agentEnv.RHQ_AGENT_JAVA_HOME = /usr/java/jdk1.6.0_25
agentMachine[1].agentEnv.RHQ_AGENT_JAVA_OPTS = -Xms150M -Xmx150M -Djava.net.preferIPv4Stack=true
agentMachine[1].agentEnv.RHQ_AGENT_ADDITIONAL_JAVA_OPTS = -Don.perftest.scenario=configurable-5 -Don.perftest.server-d-count=3 -Don.perftest.service-d-metrics-count=120 -Drhq.trace-command-config=true
### RHQ Server Configuration Properties (will get copied into each Server's rhq-server.properties file)
# Database
rhq.server.database.connection-url=jdbc:postgresql://perfdb.example.org:5432/rhq
rhq.server.database.driver-class=org.postgresql.Driver
rhq.server.database.xa-datasource-class=org.postgresql.xa.PGXADataSource
rhq.server.database.user-name=rhqadmin
rhq.server.database.password=1eeb2f255e832171df8592078de921bc
rhq.server.database.type-mapping=PostgreSQL
rhq.server.database.server-name=127.0.0.1
rhq.server.database.port=5432
rhq.server.database.db-name=rhq
hibernate.dialect=org.hibernate.dialect.PostgreSQLDialect
# Quartz
rhq.server.quartz.driverDelegateClass=org.quartz.impl.jdbcjobstore.PostgreSQLDelegate
rhq.server.quartz.selectWithLockSQL=SELECT * FROM {0}LOCKS ROWLOCK WHERE LOCK_NAME = ? FOR UPDATE
rhq.server.quartz.lockHandlerClass=org.quartz.impl.jdbcjobstore.StdRowLockSemaphore
the remote agent install code in server-jar makes SSH calls using the JSch lib (see SSHInstallUtility) - this can be copied/reused
package the utility as an executable jarjar or uberjar, so it will be neatly contained in a single jarfile, and no shell scripts, batch files, or maven assembly scripts will need to be written
look at the agentcopy.properties file to get a feel for how it does the configuration of all the agents. that might spark some additional ideas